Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly orient (y, x) arrays #1095

Merged
merged 1 commit into from
Jun 16, 2022
Merged

Correctly orient (y, x) arrays #1095

merged 1 commit into from
Jun 16, 2022

Conversation

ianthomas23
Copy link
Member

Candidate fix for #1054. Surprisingly small amount of code changed in the end.

Canvas.raster and tf.shade both accept xr.DataArrays with dims[-2:] == ('y', 'x') and the direction of each of x and y coords may be increasing or decreasing. The orientation of the DataArray returned from these functions should be the same as the input, so if e.g. the input to Canvas.raster is flipped in the x-direction then so will the output be. This keeps the data and coordinates in sync. This was not previously the case, the data was flipped within datashader but the coordinates weren't.

Because the data and its coordinates are synchronised, you can plot the DataArrays in holoviews or hvplot and they are correctly rendered with increasing x and y.

Here are two examples of this working. Firstly the OP's test case of issue #1054 using a GeoTIFF obtained from https://github.com/mommermi/geotiff_sample. The land is on the southeast, the x-coords are increasing and the y-coords are decreasing:

two

At the top the datashader Image has y-flipped as the input data does because it just draws the image data and ignores the coordinates. Below this it is plotted directly by hvplot and also using datashader which use the coordinates correctly.

Second example to be completely explicit.

one

There are 4 DataArrays, one for each of the 4 x/y flipping combinations. hv.Image and shade(image) draw them all correctly as the coordinates remain correctly oriented with respect to the data regardless of if they are passed through datashader or not. At the bottom are datashader.Images that are returned from ds.Canvas.raster followed by ds.tf.shade. The images are shown in their flipped orientation as the coordinates are ignored for rendering.

One thing I have not done is consider DataArrays with dims == ('x', 'y'). These are allowed but always interpreted as dims == ('y', 'x') so strange things can result. We will need to do something about this as even some of the reproducers in issue #1054 use dims == ('x', 'y') which made them not very helpful in diagnosing the cause of the problem. I don't think we can forceably transpose them to get ('y', 'x') but we could do some checking and either raise an exception or print a warning. But we need to be careful as we cannot just look for x and y but also need to consider lat, lon, latitude and so on.

I have kept the calc_res function as it is. This returns the x and y resolutions (i.e. spacings) of the specified DataArray, but it chooses to reverse the yresolution. This annoys me a lot as it is unnecessary, but I have left it in case of external libraries that are using this function as it is considered public.

@codecov
Copy link

codecov bot commented Jun 15, 2022

Codecov Report

Merging #1095 (432adf5) into master (de63b66) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1095   +/-   ##
=======================================
  Coverage   83.36%   83.37%           
=======================================
  Files          34       34           
  Lines        7491     7495    +4     
=======================================
+ Hits         6245     6249    +4     
  Misses       1246     1246           
Impacted Files Coverage Δ
datashader/transfer_functions/__init__.py 86.61% <100.00%> (ø)
datashader/utils.py 72.78% <100.00%> (+0.37%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de63b66...432adf5. Read the comment docs.

@maximlt
Copy link
Member

maximlt commented Jun 15, 2022

Thanks Ian! I haven't dived into the changes but I remembered that some issues might have been related to this bug so tested them with this PR as it is:

Sorry for the distraction!

@ianthomas23
Copy link
Member Author

@maximlt I am happy to have accidentally fixed another issue! I claim double credit!

I am not surprised that the polygon/matplotlib issue is not affected as I don't think the polygon code touches anything that this PR changes.

@ianthomas23
Copy link
Member Author

I think we can scratch my concerns about dims=('x', 'y') as the changes here deal with it in a reasonable way. Now that datashader returns DataArrays with the correct coordinates, flipped or not, holoviews renders them correctly regardless of dim order. Here is the second example above but using dims=('x', 'y') rather than ('y', 'x'):

xy

Holoviews output is correct. Datashader output as images ignores the attached coords so is displayed using the conventional increasing ('y', 'x'). This can give you a headache if you are trying to interpret it as shown, but if you pass it on to any other code that cares about the dims then everything turns out fine.

@philippjfr
Copy link
Member

Very glad to see this and thanks for the detailed writeup. Also great to see that we didn't have any tests that were testing the wrong result. Overall I'd be in favor of merging this asap and then tagging a dev release so we can test this more easily.

@ianthomas23
Copy link
Member Author

Self merging so we can try it out in dev release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants